Skip to content

Model updates#85

Open
ericc59 wants to merge 7 commits intomainfrom
model-updates
Open

Model updates#85
ericc59 wants to merge 7 commits intomainfrom
model-updates

Conversation

@ericc59
Copy link
Copy Markdown
Collaborator

@ericc59 ericc59 commented Mar 19, 2026

Summary

  • Add new model configs: GPT 5.4 (pro/base/mini/nano), Grok 4.20, Qwen 3.5, DeepSeek v3.2, GLM-5, and older baselines (gpt-4o, gpt-4, gpt-3.5-turbo, claude-3-haiku)
  • Add make score, make upload, and make run-benchmark Makefile targets with score_table.py CLI
  • Fix resume logic to requeue failed/in-progress tasks that have no submission file on disk
  • Extract _prepare_responses_kwargs() in OpenAI base adapter: default store=False, map reasoning_effort config to Responses API reasoning.effort, fix response parsing to skip reasoning items
  • Filter HF uploads to JSON only to prevent checkpoint files from getting uploaded (allow_patterns=["*.json"])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant